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Foreword 



This booklet is intended to be used by administrators, 
project directors, teachers, or any other educator who is 
interested in obtaining information that can be used to deter- 
mine if what \s planned as an instructional program is actually 
implemented. This, of course, must be carried out if educa- 
tional accountability is to be accomplished. 

The content of the booklet describes a format for moni- 
toring an instructional program through the use of perform- 
ance objectives and specifying how variables such as organi- 
zation, content, method, and facilities are going to be used 
to bring about the accomplishment of these performance 
objectives. 

In addition, a list of procedures is outlined describing 
how one could effectively implement this monitoring system 
in his own given educational situation. 



Preface 



Many of the ideas incorporated into this booklet were 
developed and implemented through a cooperative effort with 
the staff of John Davidson Elementary School, located in the 
Vallejo City Unified School District, Vallejo, California. 

EPIC Diversified Systems would like to express its sin- 
cere appreciation for the opportunity of working with such a 
motivated and interested staff in terms of being "account- 
able" for their instructional program. 

Specifically, the following should receive individual rec- 
ognition for much of the content of this booklet: 

Mr. Doug Stellner, Principal 

John Davidson Elementary School 

Mr. John Carter, Teacher 

John Davidson Elementary School 

Dr. Guillermo Lopez, Associate Superintendent 
Vallejo City Unified School District 
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Introduction 



Educators are faced with undertaking the tremendous 
task of ariswering the call for accountability. "Accountability” 
poses the questions: (1) what is desired with respect to behav- 
ioral changes in children as a result of participating in an 
instructional program, and (2) how effective is the instruc- 
tional program in bringing about these behavioral changes? 

Basically, three tasks must be completed in order for 
accountability to occur: 

1. Specification of the desired student performance. 

2. Establishment of procedures to monitor and audit 
the instructional program which will be implemented 
to bring about the expected student performance. 

3. Publication of a report relating student performance 
to the implemented instructional program. 

A first task is to state the desired student performance 
that will take place in a given period of time. Usually, this 
student performance is specified in the form of a performance 
objective and the period of time is usually geared to the 
school year and/or semester. A performance objective should 
contain the following six elements.’ 

1. Who is going to perform the specified behavior. 

2. The behavior that is expected to occur. 

3. The situation in which the behavior will be observed. 

4. How the behavior is going to be measured. 



1 Robert J. Armstrong ef a/. Developing and Writing Behavioral Oblecllves. 
Tucson, Arizona: Educational Innovators Press, 1970. 
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5. The expected proficiency level. 

6. The time needed to bring about the behavior. 

Below is an example of a performance objective contain- 
ing the six elements: 

By the end of the school year, all first grade students 
will demonstrate their ability to apply mathematical 
concepts as measured by an average 1.5 grade level 
increase on the CTBS. 

Given the appropriate type of in-service training, behav- 
ioral objectives containing the above elements can be devel- 
oped and stated by educators. These objectives fulfill the first 
task of bringing about accountability. 

The second task, “establishing procedures for monitor- 
ing and auditing,” the instructional program, to be imple- 
mented in order to bring about the desired behavioral changes 
has been ignored in the past. 

Consequently, many invalid conclusions have been drawn 
in' the past about the effectiveness of various instructional 
programs due to the fact that what was actually implemented 
is different from what was originally planned. 

An example of a planned program that was different 
from the actual program occurred a number of years ago 
when EPIC Diversified Systems was evaluating a sixteen-week 
teacher in-service program. 

The in-service program dealt with the training of a group 
of teachers in the use of a selected set of counseling and 
guidance techniques. It was hypothesized that if the teachers 
internalized the content of the in-service program, they would 
change their behavior in the classroom in order to (1) increase 
their recognition of student feelings, (2) increase their use 
of praise and encouragement, (3) increase their use of student 
ideas, and (4) decrease their use of criticism. The counselors 
who were conducting the in-service program specified that 
they planned to spend four weeks on each of the above behav- 
iors for a total of sixteen weeks. Since this program occurred 
before monitoring became an integral part of the evaluation 
process, no monitoring procedures were specified for the 

sixteen weeks of in-service. 

1 



2 



6 



Prior to the in-service training, the teachers were 
observed in their classrooms and statistics were compiled 
with regard to their use of these four behaviors. At the com- 
pletion of the in-service, the teachers were again observed 
and again statistics were compiled on their use of the four 
behaviors to determine if any changes had taken place. 

When the results of the classroom observations were 
evaluated, it was determined that a significant increase had 
taken place in the teachers’ recognition of student feelings, 
but there were no changes in the other three teacher behav- 
iors. Therefore, it was concluded that the in-service program 
was successful in bringing about only one of the desired 
changes. 

However, sometime after the evaluation report had been 
published, it was discovered that the counselors had actually 
implemented an in-service program where they spent fourteen 
weeks training the teachers in counseling techniques on 
recognizing student feelings and two weeks on the other three 
behaviors. At this point, the results of the in-service began to 
make sense and appeared to be what one would expect. But, 
it should be noted that some invalid conclusions were orig- 
inally drawn. 

Therefore, when one attempts to relate the degree of 
attainment of a behavioral objective to the instructional pro- 
gram, an almost impossible task exists if no monitoring of the 
program has been conducted. 

The task of establishing monitoring procedures must be 
completed in order for accountability to take place. This book- 
let will outline a set of monitoring procedures which can be 
used by educators to accomplish this task. 



Establishing a 
Monitoring System 

These procedures will provide detailed information which 
will (1) define what a given group of students actually experi- 
ences in terms of an instructional program, and (2) provide 
insights as to what changes are needed in the instructional 
program to bring about an increased attainment of a given 
set of objectives. 

To establish a sound monitoring system, all the variables 
that interact within an instructional program should be peri- 
odically observed and described. A suggested format for a 
basic monitoring system is illustrated in Figure 1. 

The format provides space for statements of the objec- 
tives that will be monitored during a predetermined time 
interval such as a weekly or monthly basis. These objectives 
are what might be called "interim performance objectives” 
because they indicate various behavioral changes which are 
desired on the part of the learner. If these interim behavioral 
changes are achieved, it would lead to the accomplishment 
of the behavioral change(s) specified in the overall perform- 
ance objective (task '*'1, accountability). 

Figure 2 showc an example of what one teacher might 
plan and implement in terms of interim performance objec- 
tives during the first week of school in working toward the 
following overall performance objective (task ’’*1, account- 
ability). 

By June 1971, those first grade students who have 
completed the total school year 1970-71 will show 
on the average a 1.5 grade level increase in read- 
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(Front Side) 

Figure 1 

MONITORING SYSTEM FORMAT 

INTERIM PERFORMANCE OBJECTIVES 

Time Interval: Grade 

Teacher 

Date 

Objectives: 



(Back Side) 

PLANNED PROGRAM 





Instructional Variables 


Organization 


Content 


Method 


Facilities 


Teacher 
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Educational 
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Family 











ACTUAL PROGRAM 
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Objectives Completed: Objectives NOT Completed: 



Objective No Revised: New No. 

Comments: 
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ing between pre/post administrations of the Stanford 
Achievement Test. 

When an instructional program is monitored, it is neces- 
sary to state performance objectives that specify the end 
products of the instructional program, and it is also necessary 
to specify what is expected in terms of learner behavior during 
the implementation of the instructional program. Not only is 
the “end product” defined (performance objective), but also 
the "path” or means of getting there is described (interim 
performance objectives). 

The reverse side of the format provides space where (1) 
the planned program can be described according to organiza- 
tion, content, method, and facilities,^ (2) a description of what 
was actually implemented, which in many instances could be 
different from that originally planned, and (3) who is respon- 
sible. This type of monitoring system will provide information 
about discrepancies that might exist between planned and 
actual programs. 

Before the procedures for using the monitoring system 
are outlined, it might be helpful to the reader if the variables 
of the system are defined. 

Students may be described in a variety of ways. 
When developing educational programs, it is essential 
that the characteristics be thoroughly described (i.e., 
age, sex, achievement level, etc.). 

The teacher variable in an educational program can 
be described in terms of grade level taught, highest 
degree held, number of years experience, etc. 

The administrator is described as the person 
usually responsible for the educational program (i.e., 
superintendent, principal, director). 

The educational specialist is described as a person 
who performs a role unique only to a particular instruc- 
tional setting (i.e., a curriculum coordinator, special 
teacher, etc.). 



2 A Scheme tor Evaluation and An Organizational Structure ot Variables. 
Tucson, Arizona: Educational Innovators Press, 1970, pp. 22-25. 



Figure 2 

MONITORING SYSTEM 



Time Interval Weekly 



Grade First 

Teacher Smith 

Date September 11,1971 



INTERIM PERFORMANCE OBJECTIVES 



1. At the end of five schooi days, first grade students wiii 
demonstrate their knowiedge of handedness (ieft-right) on 
the average of 90% of the time as measured by teacher 
observation during the piaying of the game, "Simon Says." 



2, At the end of five schooi days, first grade students wiii 
demonstrate knowiedge of upper and iower case fetters of 
the aiphabet as indicated by a minimum average score of 
90% on a teacher-constructed aiphabet test. 



3. At the end of five schooi days, first grade students wiii 
demonstrate their knowiedge of "iarger” and "smaiier” by 
correctiy answering 90% of the teacher’s questions posed 
during the use of the EDL readiness materiais. 



4, At the end of ten schooi days, first grade students wiii 
demonstrate their knowiedge of initiai and finai consonants 
of "wh,” "th,” "sh," "ch,” initial sounds and "sh" and "ch” 
final sounds by achieving a minimum average score of 90% 
on the appropriate SR A Word Games Test. 



5. At the end of twenty-one school days, first grade students 
will demonstrate their knowledge of vocabulary found in 
the first Primary Primer by achieving a minimum average 
score of 90% on a teacher-made test. 
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The family variable refers to the parents or legal 
guardian of the child, and includes brothers and sisters. 

The organization variable refers to the manner and 
time in which students are organized for learning 
(i.e., self-contained classroom, departmentalized, non- 
graded, etc.). 

The content variable is defined as a body of knowl- 
edge topically described (i.e., algebra, American prob- 
lems, reading). 

The method variable can be described as: 

(1) Teaching activities (i.e., lecture demonstration) 

(2) Types of interaction (i.e., teacher-student, stu- 
dent-student) 

(3) Learning principles or theories (i.e., operant 
conditioning) 

Figure 3 illustrates what might have been planned and 
what was actually implemented for the objectives in Figure 2. 

If this monitoring system is to be used effectively, some 
general procedures should be followed. These procedures are 
listed in sequential order. 

General Procedures of Monitoring 

step 1. Specify the overall performance objectives to be 
accomplished through a given educational program. 
The performance objectives would complete the first 
task for accountability (see pages 1 and 2 for 
accountability tasks). 

Step 2. Specify the time interval for monitoring. The instruc- 
tors that are involved in the instructional program 
should agree on the interval of time that will pass 
between monitoring points. In most instances, the 
monitoring period would probably be weekly or 
bi-weekly. 

Step 3. Select a person to be responsible for keeping a rec- 
ord of each instructor’s sheets for each monitoring 
point. This person’s responsibility would be to make 
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Instructional Variables 


Facilities 


Same, except the 
film was not used 








Method 


Same 








Content 


Same 








Organization 


30 minutes per day 
per group of 10 
students — three 
groups (obj. 1-3) 

15 minutes per day 
total class (obj. 4 & 5) 










Teacher 


Administrator 


Educational 

Specialist 


Family 
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Objectives Completed: 1,2,3 Objectives NOT Completed: 4, 5 

Objective No. Revised: New No. 

Comments: 



sure that each sheet is completely filled out — state- 
ment of objectives that will be dealt with during the 
monitoring interval, completion of Planned Program 
section at the beginning of the monitoring interval, 
completion of Actual Program section at the end of 
the monitoring interval, indication of what objectives 
were met and what objectives were not met. 

In addition to general procedures, there are some indi- 
vidual instructor procedures which should be followed if this 

monitoring system is to provide relevant feedback. The proce- 
dures are listed in sequential order. 

Individual Instructor Procedures 

step 1. Prepare interim behavioral objectives in a sequential 
order of accomplishment as they relate to the over- 
all performance objective(s). These objectives should 
include the six criteria described on pages 1 and 2. 
The only difference between interim objectives and 
the overall performance objective is that there will 
be a shorter time interval and the situation under 
which the behavior will be observed will be more 
specific. 

Step 2. Select those objectives that will be considered forthe 
given monitoring period. 

Step 3. Complete the Planned Program as it relates to the 
accomplishment of the objectives identified in Step 
2 above. 

Step 4. At the end of the monitoring interval, complete the 
Actual Program section and indicate which objec- 
tives were completed and which objectives were not 
completed. 

Step 5. Recycle by selecting the next objective(s) as identi- 
fied in Step 1 above and complete the Planned Pro- 
gram section on the next monitoring sheet. 

Step 6. Continue process. 
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Summary 



The final activity in bringing about accountability is the 
preparation of a final report that will be made public. With the 
information and data collected with a monitoring system as 
described above, this final report will be very objective in that 
it will be able to show the relation between the level of attain- 
ment of objectives and the implemented instructional pro- 
gram, using valid and reliable information. This effort would 
satisfy many public demands for accountability in education 
because the educators will now be able to say what they are 
doing to educate children and how well they are doing it. 
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